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Abstract 


This  report  describes  a  study  conducted  by  Micro  Analysis  and  Design,  Inc.,  for  the  U.S. 
Army  Research  Laboratory  (ARL).  One  area  of  research  examined  by  ARL  was  the  staffing 
required  to  operate  tactical  unmanned  aerial  vehicles  (TUAVs).  The  primary  objective  of 
the  study  was  to  use  simulation  modeling  to  analyze  how  fatigue,  crew  size,  and  rotation 
schedule  affect  operator  workload  and  performance  during  the  control  of  a  TUAV. 
Computer  simulation  models  were  developed  with  the  Micro  Saint  Discrete  Event 
Simulation  software  to  simulate  the  tasks  that  operators  perform  when  controlling  a 
TUAV.  These  models,  which  contain  system-specific  attributes  of  the  Shadow  200 
TUAV,  included  a  fatigue  function  to  predict  performance  effects  for  day  and  night 
missions.  Subject  matter  experts  (SMEs)  provided  the  list  of  tasks  involved  in  controlling  a 
TUAV  (during  normal  operations  and  emergencies),  the  order  of  these  tasks,  and  the  visual, 
auditory,  cognitive,  and  psychomotor  workload  values  associated  with  each  task.  Twelve 
different  crew  configurations  were  examined  for  the  tactical  operations  center  (TOC)  and 
the  launch  and  recovery  station  (LRS),  which  ranged  in  size  from  8  to  15  crew  members. 
The  conclusions  from  executing  the  models  and  interviewing  SMEs  (during  12-  and  18-hour 
missions)  indicate  that  reducing  the  number  of  aerial  vehicle  operators  (AVOs)  and  mission 
payload  operators  (MPOs)  in  the  TOC  can  result  in  more  aerial  vehicle  mishaps  during 
emergencies,  increased  search  time,  and  a  decreased  number  of  targets  detected.  For 
example,  compared  to  six  AVOs  or  MPOs  in  the  TOC,  the  addition  of  two  crew  members 
resulted  in  only  slight  performance  gains  of  a  6%  increase  in  target  detection  and  a  4% 
decrease  in  target  search  time.  However,  when  the  members  of  the  crew  were  reduced  to 
four  AVOs  or  MPOs  in  the  TOC,  there  was  substantial  performance  loss  (20%  decrease  in 
target  detection  and  a  15%  increase  in  target  search  time).  The  general  conclusion  is  that  a 
crew  of  12  (TOC  [two  MCs  and  six  AVOs  or  MPOs];  LRS  [two  MCs  and  two  AVOs])  is 
the  most  efficient  trade-off  between  performance  and  crew  size.  The  implications  of  these 
findings  for  other  possible  crew  configurations  are  discussed,  along  with  plans  for  further 
analyses. 
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a  mapping  exercise  that  took  place  at  Fort  Huachuca  in  March  2000. 
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USING  SIMULATION  MODELS  TO  ANALYZE  THE  EFFECTS  OF  CREW 
SIZE  AND  CREW  FATIGUE  ON  THE  CONTROL  OF  TACTICAL 
UNMANNED  AERIAL  VEHICLES  (TUAVs) 


1.  Background 


This  report  describes  a  study  conducted  by  Micro  Analysis  and  Design,  Inc.,  for  the  U.S.  Army 
Research  Laboratory  (ARL).  One  area  of  research  examined  at  ARL  is  the  staffing  required  to 
operate  tactical  unmanned  aerial  vehicle  (TUAV)  systems.  TUAVs  are  intended  to  provide 
reconnaissance,  surveillance,  and  target  acquisition,  as  well  as  battle  management  and  battle 
damage  assessment  information  in  a  50-  (threshold)  to  200-  (objective)  kilometer  radius  of  action 
from  a  controlling  ground  control  station.  TUAVs  are  to  be  deployed  worldwide  and  must  be 
capable  of  penetrating  and  successfully  operating  within  enemy  air  space  during  day  and  night 
missions  and  in  adverse  weather. 

The  primary  objective  of  the  study  was  to  use  simulation  modeling  to  analyze  how  fatigue,  crew 
size,  and  rotation  schedule  affected  operator  workload  and  performance  during  the  control  of  a 
TUAV.  A  modeling  tool  such  as  Micro  Saint  allows  analysts  to  investigate  a  wide  variety  of 
possible  future  missions  during  variable  fatigue  and  circadian  conditions.  The  results  of  this 
study  will  be  used  by  ARL  scientists  to  make  preliminary  staffing  recommendations  for  TUAV 
operational  testing.  The  importance  of  setting  realistic  crew  sizes,  based  on  actual  operational 
contingencies,  makes  the  use  of  soldier  performance  modeling  particularly  cost  effective.  Data 
from  future  field  experiments  can  be  incorporated  into  the  basic  modeling  environment  to 
validate  and  extend  the  preliminary  results. 

The  following  lists  the  four  tasks  that  represent  the  technical  objectives  of  the  project: 

1.  Develop  computer  simulation  models  that  will  address  the  impact  of  crew  size  and 
task  allocation  on  the  control  of  a  TUAV.  This  will  include  the  use  of  task  network  modeling  to 
examine  the  effects  of  varying  shift  length,  rotation  schedule,  type  of  scenario,  and  task 
allocation  on  system  performance  time  and  workload. 

2.  Review  the  literature  and  ongoing  projects  to  develop  an  algorithm  to  address  the 
effects  of  fatigue  on  performance.  Consider  the  joint  impact  of  fatigue  and  circadian  rhythm. 
Apply  this  algorithm  to  the  simulation  model.  Evaluate  the  results  to  determine  whether  various 
shift  lengths,  rotation  schedules,  or  task  allocation  strategies  can  be  used  to  reduce  the  impact  of 
these  stressors. 

3.  Perform  on-site  data  collection  and  support.  Collect  data  with  experienced  TUAV 
subject  matter  experts  (SMEs)  to  establish  crew  schedules,  rotations,  workload  values,  and 
fatigue  effects. 
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4.  Modify  the  model  so  that  it  represents  the  Shadow  2001  TUAV.  Exercise  the  model 
so  that  unique  aspects  of  the  platform  can  be  studied.  Exercising  the  model  will  include  the 
required  and  objective  operating  tempo  for  the  specific  platform  chosen  to  include  mission 
capabilities  for  these  requirements  as  defined  by  the  operational  requirements  document. 

Computer  models  were  developed  with  Micro  Saint  to  simulate  the  tasks  that  operators  perform 
when  controlling  a  TUAV.  These  models,  which  contained  system- specific  attributes  of  the 
Shadow  200  TUAV  (AAI  Corporation,  2000),  included  a  fatigue  function  to  predict  performance 
effects  for  day  and  night  missions.  SMEs  provided  the  list  of  tasks  involved  in  controlling  a 
TUAV  (during  normal  operations  and  emergencies),  the  order  of  these  tasks,  and  the  visual, 
auditory,  cognitive,  and  psychomotor  workload  values  associated  with  each  task.  The  TUAV 
models  were  developed  to  simulate  18-hour  missions  with  2-,  3-,  4-,  and  6-hour  rotation 
schedules  and  were  re-run  to  simulate  two  12-hour  missions.  Different  weather,  terrain,  search, 
and  emergency  conditions  were  also  programmed  into  the  models. 


2.  Methods 


2.1  SME  Participants 

Eighteen  SMEs  from  Fort  Huachuca,  Arizona,  provided  (a)  a  list  of  tasks  involved  in  controlling 
a  TUAV  (during  normal  operations  and  emergencies),  (b)  the  order  of  these  tasks,  (c)  the  visual, 
auditory,  cognitive,  and  psychomotor  workload  values  associated  with  each  task,  (d)  the  types  of 
emergencies  that  can  occur  during  missions,  and  (e)  the  probabilities  of  mishaps  occurring 
during  emergencies  when  soldiers  are  fatigued.  These  data  were  obtained  through  questionnaires 
and  informal  interviews.  The  SMEs,  who  were  made  available  to  this  project  by  the  Training  and 
Doctrine  Systems  Manager  (TSM)  for  TUAVs,  represented  the  best  available  civilian  and 
military  expertise  available  in  the  Army.  At  the  time  the  interviews  were  conducted,  Fort 
Huachuca  was  the  national  training  school  for  all  Department  of  Defense  unmanned  aerial 
vehicle  (UAV)  operators.  In  addition,  these  data  were  supplemented  with  information  obtained 
from  the  operational  model  summary  and  mission  profile  for  the  Shadow  200  and  from  previous 
TUAV  models  and  studies  (Bames,  Knapp,  Tillman,  Walters,  &  Velicki,  2000). 

2.2  Modeling 

Simulation  models  developed  on  computers  can  be  used  to  simulate  operator  and  crew 
performance  to  fill  gaps  in  knowledge  and  to  develop  regulatory  guidance.  Depending  on  the 
parameters  of  performance  modeled  (e.g.,  objective  performance,  cognitive  performance,  etc.), 
these  simulations  often  permit  interaction  between  operators  in  the  models  and  the  simulation 
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context  (e.g.,  the  command  center),  between  operators  and  objects  (e.g.  procedures),  and 
between  operators  (i.e.,  intra-crew  activities). 

The  objects  and  operators  modeled  in  such  simulations  have  parameters  that  define  how  they 
function  in  the  simulation.  Consider,  for  example,  a  simulation  of  an  operator’s  performance  in 
following  procedures  to  achieve  a  mission  objective.  The  model  of  the  operator  can  include  such 
parameters  as  the  operator’s  previous  experience  and  familiarity  with  the  operation,  the 
accessibility  of  the  procedures  both  physically  and  conceptually,  the  number  of  tasks  and  steps, 
the  relative  complexity  of  the  task,  and  the  likelihood  of  error.  Parameters  of  the  procedure(s) 
can  be  modeled  to  predict  how  qualitative  aspects  of  the  procedures  (e.g.,  readability, 
complexity,  and  format)  may  affect  operator  performance.  Parameters  of  the  procedures  can  also 
be  modeled  to  predict  how  relevant  aspects  of  the  command  center  (e.g.,  location,  operating 
mode,  and  condition  complexity)  affect  operator  performance. 

Because  models  are  run  on  computer  systems,  there  are  no  practical  limitations  on  the  number  of 
trials  or  simulation  runs  that  can  be  made.  Models  can  be  programmed  to  include  provisions  for 
user-input  values  for  parameters  and  rules  (e.g.,  number  of  operators,  type  of  interface,  and  time 
of  day),  thereby  allowing  different  performance  conditions  to  be  simulated.  This  permits  the 
investigation  of  possible  changes  in  operator  or  crew  performance  that  can  occur  in  response  to 
changes  in  other  aspects  of  the  simulated  context  and  objects.  Changes  include  the  timing  of 
events,  variation  of  performance  shaping  factors  (e.g.,  stress,  time  available,  quality  of 
procedures,  and  crew  interaction  quality)  and  the  availability  of  crew  members  (i.e.,  whether 
normally  staffed  or  minimally  staffed).  Because  models  do  not  have  the  problems  that 
researchers  have  in  finding  available  crews  to  be  research  subjects  and  because  models  allow 
users  to  vary  parameters,  simulations  represent  a  resource  for  generating  information  about 
operator  and  crew  performance  during  statistically  high  power  conditions. 

2.3  Discrete  Event  Simulation  With  Micro  Saint 

Discrete  event  simulations  (DES)  use  a  computer  model  to  describe  a  process  that  can  be 
expressed  as  a  sequence  of  events,  each  with  a  distinct  beginning  and  end.  Events  can  be  any  part 
of  the  process  such  as  scheduled  activities  or  tasks  that  represent  the  flow  of  the  process.  The 
tasks  are  displayed  schematically  on  a  diagram  called  the  task  network  diagram,  which  is  the 
basis  of  the  model. 

Micro  Saint  is  a  simulation  software  package  for  constructing  models  that  simulate  real-life 
processes.  These  models  can  be  relatively  simple  or  complex.  One  can  build  a  simple,  functional 
model  by  creating  a  network  diagram  and  entering  task-timing  information  for  each  task  in  the 
network.  More  complex  models  can  be  built,  which  include  dynamically  changing  variables, 
probabilistic  and  tactical  branching  logic,  conditional  task  execution,  and  extensive  model  data 
collection.  One  can  specify  all  these  by  choosing  menu  commands  or  by  providing  expressions 
for  Micro  Saint  to  execute  during  specific  circumstances. 
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Whether  the  model  is  simple  or  complex,  the  process  of  executing  a  Micro  Saint  model  and 
generating  statistics  and  graphs  from  the  collected  data  is  mostly  automatic.  In  addition,  the 
results  files  can  be  opened  in  spreadsheets  or  statistical  packages  for  further  analysis.  For 
questions  and  a  more  detailed  understanding  of  Micro  Saint,  refer  to  the  Micro  Saint  User’s 
Manual  (Micro  Analysis  and  Design,  1999). 

2.4  Fatigue 

The  impact  of  fatigue  induced  by  sleep  deprivation  or  poor  sleep  has  always  played  an  important 
role  on  the  effectiveness  of  troops  (Belenky,  Kreuger,  Bailking,  Headley,  &  Solick,  1987).  The 
impact  is  not  lessened  by  the  increase  in  night  operations  and  high  intensity  around-the-clock 
operations  expected  in  near  future  threats.  Fatigue  is  arguably  one  of  the  most  persistent  threats 
to  mission  success  during  sustained  or  continuous  operations.  Much  is  known  about  the  effects 
that  fatigue  has  on  performance.  The  effects  have  been  consistently  demonstrated  in  multiple 
sleep  deprivation  studies.  For  example,  cognitive  capacity  can  decline  about  60%  after  two 
nights  of  sleep  deprivation  (Angus  &  Heselgrave,  1985).  Decreases  in  asymptotic  performance 
between  10%  and  20%  are  reported  during  an  extended  performance  of  less  than  24  hours’ 
duration  (Benline,  French,  &  Wing,  1997).  Recently,  Dawson  and  Reid  (1997)  proposed  that 
performance  after  21  hours  of  sleep  deprivation  was  comparable  with  performance  degradation 
following  legal  levels  (0.1%)  of  intoxication. 

The  fatigue  algorithm  used  for  this  project  predicts  human  response  capability  for  tasks  over  an 
extended  period  of  sleep  deprivation  (as  many  as  52  hours).  The  main  focus  of  the  algorithm  is 
the  interaction  of  sleep  deprivation  (i.e.,  fatigue)  with  circadian  disruption  on  performance.  It  is 
based  on  data  collected  at  Brooks  Air  Force  Base,  San  Antonio,  Texas,  with  pilots  as  subjects 
during  a  sleep  deprivation  study.  More  than  15  different  tasks  were  used  in  the  study,  and  the 
subjects  cycled  through  each  about  once  every  1.5  hours.  The  fatigue  algorithm  is  based  on  one 
of  those  tasks:  the  Maniken  task  of  the  attention-switching  task.  This  task  was  selected  because 
it  is  a  complex  visual  task;  it  required  the  subject  to  pay  attention  to  a  signal  on  the  screen  while 
he  performed  one  of  two  tasks  to  know  when  to  switch  to  and  from  one  task  to  another.  It  is  an 
intellectually  challenging  test  that  has  consistently  proved  sensitive  to  fatigue  and  other  stressors 
in  a  number  of  experiments.  It  is  similar  to  the  kinds  of  visual  and  performance  demands  placed 
on  TUAV  operators. 

The  Brooks  Air  Force  data  were  plotted  as  shown  in  Figure  1.  A  cosine  curve  was  fit  to  the  data 
to  unmask  the  circadian  features  of  performance  (Naitoh,  Englund,  &  Ryman,  1985).  This 
involved  a  complex  demodulation  function  to  separate  the  linear  aspects  of  the  data.  The 
remainder  is  the  circadian  function  that  allows  oscillating  performance  levels  to  be  reliably 
predicted  for  extended  periods  of  time  over  several  days  (Redmond,  Sing,  &  Hegge,  1982).  The 
algorithm  accounts  for  a  significant  amount  of  the  variance;  although  most  of  the  variance  is 
linear  (0.89%),  the  cosine  fit  provides  an  important  oscillating  function. 
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Figure  1.  The  Maniken  Data  Used  to  Make  the  Algorithm  (Mod  1),  the  Cosine  Function 
Data  (COS),  the  Linear  Function  Data  (linear),  and  the  Predicted  Response 
Time  Capability  (average)  Expressed  as  Percent  Baseline  Response  Time. 


In  the  model,  crew  size  and  rotation  schedule  affect  the  soldiers’  sleep  and  rest  cycles,  which  in 
turn  affect  their  fatigue,  circadian  rhythms,  workload,  and  performance.  As  fatigue  and  workload 
increase,  target  search  time,  target  detection  rate,  and  human  errors  increase,  thus  increasing  the 
likelihood  of  TUAV  mishaps.  Also,  the  more  time  an  operator  spends  on  a  task  (vigilance)  such 
as  monitoring,  the  greater  the  decrease  in  his  or  her  performance  over  time.  In  the  model,  the 
times  to  perform  the  tasks  were  generated  from  SMEs.  These  values  were  then  adjusted  by  the 
fatigue  algorithm,  the  time  into  the  scenario,  and  vigilance  decrements  based  on  how  much  time 
was  spent  monitoring  (Deaton  &  Parasuraman,  1988;  Molloy  &  Parasuraman,  1996).  Because 
soldiers  were  limited  to  12-hour  duties,  fatigue  played  a  minor  role  in  affecting  task  perfor¬ 
mance.  Instead,  vigilance  and  workload  had  the  most  effect  on  target  search  time  and  the  number 
of  targets  detected. 


2.5  Assumptions 

Appendix  A  provides  a  sample  of  some  of  the  task  networks  in  the  model.  The  model  simulates 
the  tactical  operations  center  (TOC)  and  the  launch/recover  station  (LRS)  (mission  commander 
[MC],  aerial  vehicle  operator  [AVO],  and  mission  payload  operator  [MPO]  duties),  and  the 
following  functions:  launch,  transfers,  recovery  of  the  TUAVs,  mission  support,  emplacement, 
displacement,  emergencies,  mishaps,  and  basic  duties  of  the  maintenance  crew  during 
emplacement.  The  model  does  not  simulate  the  platoon  sergeant,  the  platoon  leader,  or  the 
complete  maintenance  duties.  Parameters  and  assumptions  about  the  Shadow  200  and  the 
corresponding  mission  profiles  were  obtained  from  the  operational  model  summary  and  mission 
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profile  for  the  Shadow  200  and  the  mapping  exercise  that  took  place  at  Fort  Huachuca.  These 
parameters  were  used  to  help  further  define  the  structure  of  the  model. 

The  TUAV  model  simulates  12-  and  18-hour  missions  (over  a  24-hour  period)  during  15 
different  conditions  for  three  consecutive  days.  During  missions,  there  are  times  when  two  aerial 
vehicles  (AV)  are  in  flight:  when  one  AV  is  observing  the  targets  and  one  AV  is  flying  to 
assume  control  of  the  search.  Although,  the  workload  scores  were  affected  by  the  overlap  when 
two  AVs  were  in  flight  simultaneously,  the  target  detection  scores  were  not  adjusted  for  the 
overlap  period.  Soldiers  will  work  2-,  3-,  4-,  or  6-hour  rotation  schedules.  During  times  when 
they  are  not  in  the  shelter  or  moving,  soldiers  may  be  resting,  eating,  performing  guard  duty,  or 
mission  planning.  However,  the  model  does  not  simulate  specifically  what  the  soldiers  are  doing 
during  these  periods. 

The  models  simulate  one  move  (jump)  per  day  for  the  TOC  and  one  move  every  other  day  for 
the  LRS.  Each  move  consists  of  a  half-hour  “break-down,”  half-hour  move,  and  a  1-hour  setup. 
The  time  to  the  destinations  will  not  vary.  A  TUAV  spends  5  hours  of  simulation  time  in  the  air: 
4  hours  of  surveillance  and  1  hour  to  fly  to  and  from  its  destination.  The  model  simulates  three 
TUAVs  and  one  floating  TUAV.  TUAVs  are  not  flown  in  snow,  thunderstorms,  high  winds,  or 
heavy  rain  conditions.  The  output  produced  by  the  model  includes  performance  times  and  target 
detection  rates  and  AV  mishaps  during  each  of  the  simulated  conditions. 

2.6  Crew  Configurations 

The  following  crew  configurations  for  the  TOC  and  LRS  were  simulated  in  the  model.  Table  1 
lists  the  12  different  combinations  that  these  configurations  produced. 

TOC  LRS 

2  MCs  and  4  AVOs  /  MPOs  0  MCs  and  2  AVOs 

2  MCs  and  6  AVOs /MPOs  1  MC  and  2  AVOs 

2  MCs  and  8  AVOs /MPOs  2  MCs  and  2  AVOs 

2  MCs  and  3  AVOs 

Appendix  B  lists  the  rotation  schedules  used  in  the  model  for  each  crew  configuration.  Crew  size 
determines  the  rotation  schedule  because  of  the  12-hour  limit  that  is  placed  on  the  duty  shift  of 
soldiers. 
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Table  1.  Crew  Configurations  for  Soldiers  With  a  Military  Occupational  Specialty  of  96U 


Crew  for  each  12-hour  shift 

Total  crew  size 

TOC 

LRS 

8 

1  MC,  2  AVOs/MPOs 

0  MCs,  1  AVO 

9 

1  MC,  2  AVOs/MPOs 

0.5  MCa,  1  AVO 

10 

1  MC,  2  AVOs/MPOs 

1  MC,  1  AVO 

10 

1  MC,  3  AVOs/MPOs 

0  MCs,  1  AVO 

11 

1  MC,  2  AVOs/MPOs 

1  MC,  1.5  AVOsb 

11 

1  MC,  3  AVOs/MPOs 

0.5  MCa,  1  AVO 

12 

1  MC,  3  AVOs/MPOs 

1  MC,  1  AVO 

12 

1  MC,  4  AVOs/MPOs 

0  MCs,  1  AVO 

13 

1  MC,  3  AVOs/MPOs 

1  MC,  1.5  AVOsb 

13 

1  MC,  4  AVOs/MPOs 

0.5  MC,  1  AVO 

14 

1  MC,  4  AVOs/MPOs 

1  MC,  1  AVO 

15 

1  MC,  4  AVOs/MPOs 

1  MC,  1.5  AVOsb 

‘There  is  actually  only  one  MC  available  for  this  configuration. 
bThere  are  actually  a  total  of  three  AVOs  for  this  configuration. 


2.7  Conditions 

The  following  list  describes  different  conditions  that  can  affect  a  TUAV  mission.  These 
conditions  are  categorized  into  the  type  of  search  being  performed,  emergencies  that  can  occur, 
weather  conditions,  and  terrain.  Different  conditions  can  affect  the  rates  of  target  detection  and 
time  to  perform  a  task  and  fly  the  TUAV.  Emergencies  cause  operators  to  perform  specific  tasks 
that  reduce  search  time. 

•  The  type  of  search  being  performed:  area  search,  person  search,  airfield,  tanks, 
building,  road  search,  bridge,  missile  site,  command  post,  air  defense  artillery  (ADA),  check 
points,  battle  damage  assessment  (BDA)  on  surface-to-air  missile  (SAM),  artillery  search. 

•  Emergencies  that  can  occur:  icing,  generator  failure,  signal  degrada-tion  or 
intermittent  link  loss,  payload  failure,  AVO  or  MPO  console  failure,  global  positioning  satellite 
(GPS)  failure. 

•  Weather:  humidity,  sunny,  gusty  winds,  crosswinds,  flat  clouds,  ragged  clouds. 

•  Terrain:  high  vegetation,  desert  (sand),  high  desert,  city,  town  or  village. 

2.8  Workload 

Workload  values  for  each  task  in  the  model  were  obtained  from  SMEs  at  Fort  Huachuca. 
Workload  was  estimated  from  a  scale  developed  by  McCracken  and  Aldrich  (1984)  and  later 
enhanced  by  Szabo  and  Aldrich  (1987)  and  Aldrich,  Szabo,  and  Bierbaum  (1989).  Their  scale 
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was  originally  developed  to  provide  a  workload  estimate  compatible  with  Wickens  (1984),  in 
which  mental  workload  is  viewed  as  consisting  of  multiple  cognitive  resources.  The  scale  was 
originally  designed  for  use  in  discrete  task  network  tools.  Four  resources  or  components  are 
typically  used  in  mental  workload  models:  visual,  auditory,  cognitive,  and  psychomotor. 
Typically,  the  visual  and  auditory  components  refer  to  the  information  processing  of  stimuli 
surrounding  a  mission  task  event.  The  cognitive  component  consists  of  the  information 
processing  synthesis.  The  psychomotor  component  is  directed  by  the  physical  responses  required 
of  a  mission  event. 

Workload  theory  is  based  on  the  idea  that  every  task  a  human  performs  requires  some  work. 
Usually,  a  task  is  composed  of  several  different  types  of  work,  such  as  visual  or  cognitive.  For 
example,  consider  a  task  such  as  steering  a  car.  This  task  will  have  some  visual  work  (watch 
where  you  are  going),  some  cognitive  work  (decide  if  you  are  turning  enough),  and  some 
psychomotor  work  (rotate  the  steering  wheel).  The  workload  theory  implemented  in  this  effort 
assigns  values  representing  the  amount  of  effort  that  must  be  expended  in  each  channel  in  order 
to  perform  the  task.  The  scale  for  each  component  ranges  from  0  (very  low  workload)  to  7  (very 
high  workload). 

This  theory  also  hypothesizes  that  if  you  are  doing  two  tasks  at  once,  the  workload  levels  are 
additive  within  channels,  across  tasks.  For  example,  if  you  were  doing  two  tasks  at  once,  one 
with  a  psychomotor  load  of  2.6  and  one  with  a  psychomotor  load  of  4.6,  then  a  psychomotor 
score  of  7.2  (2.6  +  4.6)  would  be  recorded  for  the  time  that  the  two  tasks  were  being  performed 
together.  A  criterion  of  overload  was  set  at  7  for  each  channel  and  28  for  a  particular  task  load. 
These  settings  were  derived  from  experience  with  rotor  aircraft  and  they  reflect  an  increased 
potential  for  accidents — a  finding  that  agrees  with  experimental  results  during  UAV  missions  as 
well  (Barnes  &  Matz,  1998). 

2.9  Missions 

The  model  simulates  five  TUAV  launches  per  day  for  an  18-hour  mission,  regardless  of  crew 
rotation  schedule.  For  each  launch,  three  different  types  of  target  searches  were  performed.  This 
resulted  in  15  different  types  of  searches  per  day.  These  missions  were  repeated  every  day  for 
3  days  (72  hours)  in  the  model  for  each  crew  rotation  schedule.  Rather  than  simulate  every 
combination  of  the  four  conditions  described  earlier,  Table  2  lists  the  15  that  were  chosen,  based 
on  SMEs’  opinion  of  the  importance  of  the  predicted  results. 

To  determine  the  amount  of  power  necessary  to  detect  a  difference  in  experimental  conditions,  a 
power  analysis  table  within  Keppel  (1991)  was  referenced.  With  an  alpha  level  of  .05  and  a  small 
effect  size  (.01),  it  was  necessary  to  obtain  354  data  points  per  cell  to  obtain  90%  power.  Thus, 
the  model  was  run  400  times  for  each  rotation  schedule  to  obtain  slightly  more  than  90%  power. 


Table  2.  Conditions  (scenarios)  Simulated  in  the  Model 


Search 

Emergency 

Weather 

Terrain 

Area  search 

Engine  failure 

Sunny/high  winds 

Desert 

Personnel  search 

Intermittent  link  loss 

Sunny 

City 

Airfield  search 

Payload  failure 

Ragged  clouds 

High  vegetation 

Tank  search 

AVO/MPO  console  fails 

Cross  winds 

High  desert 

Building  search 

GPS  failure 

Hazy 

City  (2  square  blocks) 

Road  search 

AVO/MPO  console  fails 

Gusty  winds 

Desert 

Bridge  search 

Intermittent  link  loss 

Low  clouds,  humid 

Town 

SCUD  site 

Payload  failure 

Hazy/foggy 

High  desert 

Command  post 

AVO/MPO  console  fails 

Rain 

City 

ADA  check  points 

None 

Gusty  winds 

High  vegetation 

BDA  on  SAM 

None 

Humid 

Desert 

Artillery  search 

None 

Sunny 

Desert 

Area  search 

Icing 

Snow  on  ground 

Town 

Tank  search 

AVO/MPO  console  fails 

Clouds  -  flat 

Desert 

Building  search 

None 

Scattered  clouds 

High  vegetation 

3.  Results 


An  analysis  of  variance  was  performed  on  the  model  output  to  see  how  different  crew  sizes  and 
fatigue  affect  the  amount  of  time  that  operators  spend  searching  for  a  target  and  the  number  of 
targets  detected  during  a  72-hour  period.  Scores  for  the  2-  and  3-hour  rotation  schedules  were 
combined  because  they  were  not  significantly  different  from  each  other  and  because  they  used 
the  same  number  of  crew  members.  The  results  showed  a  significant  effect  for  crew  size  on  the 
model  output:  (F(3,1396)  =  12759.24;  p  <  0.01)  and  (F(3,1396)  =  7379.29;  p  <  0.05), 
respectively.  Post  hoc  analyses  (Tukey’s  Honestly  Significant  Difference  Test)  revealed  that 
decreasing  the  crew  size  results  in  an  increase  in  the  amount  of  time  to  detect  a  target  (see  Figure 
2)  and  a  decrease  in  the  number  of  targets  detected  (see  Figure  3).  The  only  comparison  that  was 
significant  (p  <  0.05)  was  between  the  crew  sizes  of  six  and  four.  Decreasing  the  number  of 
AVOs  or  MPOs  in  the  TOC  from  eight  to  six  resulted  in  a  4%  increase  in  the  time  to  search  a 
target  and  a  6%  decrease  in  the  number  of  targets  detected.  Decreasing  the  number  of  AVOs  or 
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MPOs  in  the  TOC  from  six  to  four  resulted  in  a  15%  increase  in  the  time  to  search  a  target  and  a 
20%  decrease  in  the  number  of  targets  detected. 


Total  #  of  AVOs/MPOs  in  the  TOC 


Figure  2.  The  Mean  Target  Search  Time  for  Each  Crew  Size  for  an  18-hour  Mission. 


8  6  4 

Total  #  of  AVOs/MPOs  in  the  TOC 


Figure  3.  The  Mean  Number  of  Targets  Detected  for  Each  Crew  Size  for  an  18-hour 
Mission. 


Analysis  of  the  SME  data  indicated  that  almost  three  times  as  many  AV  mishaps  can  occur  when 
a  crew  is  fatigued  than  when  they  are  well  rested  (see  Figure  4).  These  mishaps  are  related  to  a 
console  failure,  lost  link,  GPS  failure,  and/or  degraded  communications.  We  calculated  the 
number  of  mishaps  for  the  well-rested  and  fatigued  conditions  by  multiplying  the  probability  of 
an  emergency  occurring  times  the  probability  of  making  a  mistake  during  the  emergency  times 
the  probability  of  a  mishap  occurring  if  a  mistake  were  made.  The  calculated  values  for  each 
emergency  were  added  together  to  produce  the  results  in  Figure  4. 
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Figure  4.  The  Number  of  AV  Mishaps  per  1000  Missions  When  a  Crew  is 
Fatigued  or  Well  Rested. 

Analysis  of  the  workload  data  produced  by  the  model  showed  that  when  there  was  no  MC  in  the 
LRS,  the  TOC  MC  was  interrupted  approximately  50%  of  the  time  with  tasks  that  would  have 
been  performed  by  the  LRS  MC  (see  Figure  5).  (A  total  workload  value  greater  than  28 
[threshold]  indicates  an  overload  on  at  least  one  or  more  workload  channels  [see  the  discussion 
section  for  the  implications  of  exceeding  this  threshold].)  For  a  12-hour  shift,  this  means  that 
6  hours  of  the  TOC  MC’s  time  were  spent  performing  the  duties  of  the  LRS  MC.  When  there 
was  one  MC  in  the  LRS,  the  TOC  MC  was  interrupted  approximately  20%  (2.5  hours)  of  the 
time  with  tasks  that  would  have  normally  been  performed  by  the  LRS  MC  (see  Figure  6). 
Finally,  analysis  of  the  workload  data  showed  that  adding  a  third  AVO  to  the  LRS  (compared  to 
the  baseline  of  two  AVOs)  did  not  affect  the  performance  of  a  mission. 


Well  Rested  Fatigued 


TOC  MC 

TOC  MC  w/  0  LRS  MCs 


Figure  5.  Workload  Values  for  the  TOC  MC  During  a  12-hour  shift  With  No  MC  or  Two  MCs 
in  the  LRS.  (The  red  line  is  the  overload  threshold.) 
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Time  into  shift  (hours) 


Figure  6.  Workload  Values  for  the  TOC  MC  During  a  12-hour  shift  With  One  or  Two  MCs 
in  the  LRS.  (The  red  line  is  the  overload  threshold.) 


3.1  Additional  Modeling 

The  TUAV  model  was  then  adjusted  to  simulate  two  new  mission  profiles:  a  12-hour  mission 
with  1-hour  gaps  between  flights  and  a  12-hour  mission  with  no  gaps  between  flights  (see 
Appendix  C  for  the  mission  profiles).  Three  AV  launches  were  simulated  per  day  rather  than  five 
(as  were  simulated  with  the  18-hour  mission).  All  other  parameters  remained  the  same. 

3.2  Additional  Modeling  Results 

The  same  performance  trends  that  were  found  with  the  18-hour  mission  were  also  found  with 
both  12-hour  missions:  reducing  the  number  of  AVOs  or  MPOs  in  the  TOC  resulted  in  an 
increase  in  the  time  to  search  for  targets  and  a  decrease  in  the  number  of  targets  detected.  No 
performance  differences  were  found  between  the  12-hour  mission  with  1-hour  gaps  (between 
flights)  and  the  12-hour  mission  with  no  gaps  (between  flights).  Approximately  40%  fewer 
targets  were  found  during  the  12-hour  missions  than  during  the  18-hour  mission  because  of  the 
difference  in  the  number  of  AV  launches  per  day.  No  differences  in  average  target  search  times 
were  found  between  the  12-hour  missions  and  the  18-hour  mission. 


4.  Discussion 


Changes  in  military  contingencies  in  the  last  decade  have  led  to  reduced  funding  for  expensive 
field  exercises  and  training.  Consequently,  planners  have  increasingly  turned  to  modeling  and 
simulation  efforts  for  “war  gaming”  and  to  estimate  the  operational  impact  of  new  systems. 
Evaluating  the  human  impact  on  complex  systems  has  lagged  and  has  resulted  in  a  decrease  in 
system  outcome  fidelity.  Small,  computer-derived  network  simulation  models  are  ideal  for  these 
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studies  and  can  address  a  wide  variety  of  human  interface  solutions  and  the  effects  of  operator 
workload. 

Computer  models  are  frequently  used  to  estimate  theater  losses  attributable  to  nuclear,  chemical, 
or  biological  events.  However,  other  threats  may  be  amenable  to  computer  evaluation;  such 
evaluation  would  enhance  the  realism  of  modeling  or  war-gaming  exercises.  Fatigue  induced  by 
sleep  deprivation  or  by  poor  sleep  has  always  played  an  important  role  in  the  effectiveness  of 
troops.  The  impact  is  not  lessened  by  the  increase  in  night  operations  and  high  intensity,  around- 
the-clock  operations  expected  in  near  future  threats. 

ARL  is  concerned  with  the  staffing  required  to  operate  TUAVs.  The  operational  requirements  of 
the  TUAV  operators  may  include  extended  duty  days,  reduced  crew  size,  and  varying  shift 
schedules.  These  conditions  are  likely  to  reduce  operator  effectiveness  because  of  fatigue.  The 
objective  of  this  study  was  to  use  simulation  modeling  to  analyze  how  fatigue,  crew  size,  and 
rotation  schedule  affect  operator  workload  and  performance  during  the  control  of  a  TUAV. 
Twelve  different  crew  configurations,  which  ranged  in  size  from  8  to  15  crew  members,  were 
examined  for  the  TOC  and  the  LRS. 


5.  Conclusions  and  Recommendations 


The  conclusions  from  executing  the  models  and  interviewing  SMEs  (during  12-  and  18-hour 
missions)  indicate  that  reducing  the  number  of  AVOs  or  MPOs  in  the  TOC  can  result  in  more 
AV  mishaps  during  emergencies,  an  increase  in  the  time  to  search  for  targets,  and  a  decrease  in 
the  number  of  targets  detected.  For  example,  compared  to  six  AVOs  or  MPOs  in  the  TOC, 
adding  two  crew  members  resulted  in  only  slight  performance  gains  of  a  6%  increase  in  target 
detection  and  a  4%  decrease  in  target  search  time,  whereas  subtracting  two  crew  members  (four 
AVOs  or  MPOs  in  the  TOC)  resulted  in  substantial  performance  losses  of  a  20%  decrease  in 
target  detection  and  a  15%  increase  in  target  search  time.  The  general  conclusion  was  that  crews 
containing  six  AVOs  or  MPOs  were  the  most  efficient  trade-off  between  performance  and  crew 
size. 

Furthermore,  analysis  of  the  workload  data  produced  by  the  model  showed  that  when  there  was 
no  MC  in  the  FRS,  the  TOC  MC  was  interrupted  approximately  50%  of  the  time  with  tasks  that 
would  have  been  performed  by  the  FRS  MC.  When  there  was  one  MC  in  the  FRS,  the  TOC  MC 
was  interrupted  approximately  20%  of  the  time  with  tasks  that  would  have  been  performed  by 
the  FRS  MC.  According  to  SMEs,  the  following  events  can  occur  when  the  TOC  MC  is 
overloaded: 

•  Delay  of  an  AV  launch  (i.e.,  the  MC  missed  the  launch  time); 

•  The  MC  encounters  problems  coordinating  the  airspace; 
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•  The  MC  is  not  supervising  the  A  VO  and/or  the  MPO; 

•  The  MC  fails  to  set  the  correct  radio  frequency,  which  can  lead  to  the  loss  of  an  AV; 

•  The  MC  incorrectly  performs  the  dynamic  re-tasking,  and  targets  are  missed; 

•  Supervision  of  maintenance  problems  can  be  degraded; 

•  Fuel  could  be  improperly  loaded  on  the  AV ; 

•  Preventive  maintenance  checks  and  services  may  not  be  performed  or  documented 
properly; 

•  The  MC  could  ignore  poor- weather  warnings,  which  could  lead  to  the  loss  of  an  AV; 

and 

•  A  jump  may  not  be  possible  unless  another  soldier  at  the  LRS  can  perform  the  MC’s 

duties. 


Adding  a  third  A  VO  to  the  LRS  (compared  to  the  baseline  of  two  AVOs)  was  shown  not  to 
affect  the  performance  of  a  mission.  Therefore,  of  the  crew  configurations  investigated,  the  crew 
size  of  12  (TOC  [two  MCs  and  six  AVOs  or  MPOs],  LRS  [two  MCs  and  two  AVOs])  proved  to 
be  the  most  effective  during  12-  and  18-hour  missions.  This  recommendation  must  be  verified 
during  experimentation  with  actual  performance  and  fatigue  data  to  validate  the  model’s 
predictions.  Also,  we  need  to  emphasize  that  the  recommendations  depend  on  the  model’s 
assumption  of  a  12-hour  shift  and  adequate  rest  during  off-shift  hours.  Actual  combat  conditions, 
additional  stress  and  duties,  and  operations  longer  than  72  hours  may  have  further  adverse  effects 
on  the  crew’s  ability  to  control  the  air  vehicle  and  perform  the  target  taskings. 

Future  work  involves  (a)  revising  the  TUAV  model  after  data  from  operational  tests  are  received 
and  (b)  performing  a  maintainability  assessment  of  the  Shadow  200  TUAV.  The  revised  model 
will  be  used  to  predict  manpower  requirements  by  simulating  the  maintenance  requirements  of  a 
unit  as  the  systems  are  sent  on  missions,  sent  to  maintenance  (as  required),  and  then  returned  to  a 
pool  of  available  systems.  Total  crew  composition  will  not  be  known  until  this  study  is 
completed.  Total  crew  size  includes  this  information  plus  data  about  administrative  and 
supervisory  personnel. 
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TASK  NETWORK  OF  THE  MICRO  SAINT  SHADOW  200  TUAV  MODEL 


Network  0  Shadow  200 TUAV  18hr  Surge  2hr  Rotation 


j,  1300 

1 

1400 

toeconinis 

- ► 

toccomnis 

(internal) 

(external) 

Figure  A-l.  Top  Level  of  the  Shadow  200  TUAV  Model. 


Network  100  Set-up  &  Re-flight 


Figure  A-2.  Set-up  and  Pre-flight  Sub-network. 
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ROTATION  SCHEDULES 


Table  B-l.  Rotation  Schedules  for  the  TOC  (18-hour  mission  and  one  move  every  day) 


TIME 

UAV  1 

UAV  2 

UAV  3 

2  MCs 

8  AVO/MPOs 
2-Hour  Rotation 

2  MCs 

8  AVO/MPOs 
3-Hour  Rotation 

2  MCs 

6  AVO/MPOs 

2  MCs 

4  AVO/ 
MPOs 

0000 

Plan¬ 
ning  & 
Setup 

MC_1  &  Op_A 
&  Op_B 

MC_1  &  Op_A 
&  Op_B 

MC_1  &  Op_A 
&  Op_B 

MC_1  & 

Op  A  & 
Op_B 

0030 

Launch 

0100 

AOO 

Setup 

0130 

Setup 

0200 

Op C  &  Op D 

Op B  &  Op C 

0230 

0300 

Op C  &  Op D 

0330 

0400 

Op A  &  Op B 

Op C  &  Op A 

0430 

Launch 

0500 

Return 

AOO 

0530 

Land 

0600 

Setup 

Op C  &  Op D 

Op A  &  Op B 

Op A  &  Op B 

0630 

0700 

0730 

0800 

Planning 

Op A  &  Op B 

Op B  &  Op C 

0830 

Launch 

0900 

Return 

AOO 

Op C  &  Op D 

0930 

Land 

1000 

Setup 

Op C  &  Op D 

Op C  &  Op A 

1030 

1100 

1130 

1200 

Launch 

MC_2  &  Op_E 
&  Op_L 

MC_2  &  Op_E 
&  Op_L 

MC_2  &  Op_D 
&  Op_E 

MC_2  & 
Op_C& 

Op D 

1230 

AOO 

Return 

1300 

Land 

1330 

1400 

Op G  &  Op H 

Op E  &  Op L 

1430 

1500 

Op G  &  Op H 

1530 

Planning 

1600 

Launch 

Op E  &  Op L 

Op L  &  Op D 

1630 

Return 

AOO 

1700 

Land 

1730 

1800 

Op G  &  Op H 

Op E  &  Op L 

Op D  &  Op E 

1830 

1900 

Return 

1930 

Land 

2000 

TD 

TD 

TD 

TD 

2030 

Move 

Move 

Move 

Move 

2100 
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Table  B-2.  Rotation  Schedules  for  the  LRS  (18-hour  mission  and  one  move  every  other  day) 


TIME 

UAV  1 

UAV  2 

UAV  3 

0  MCs 

2  AVOs 

1  MC 

2  AVOs 

2  MCs 

2  AVOs 

2  MCs 

3  AVOs 

0000 

Planning 
&  Setup 

AVO_l 

AVO_l  & 
MC  on 

AVO_l  & 
MC 1 

AVO_l  & 
MCI 

0030 

Launch 

0100 

AOO 

Setup 

0130 

Setup 

0200 

0230 

0300 

0330 

0400 

0430 

Launch 

0500 

Return 

AOO 

0530 

Land 

0600 

Setup 

0630 

MC  Off 

0700 

AVO 2 

0730 

0800 

Planning 

0830 

Launch 

0900 

Return 

AOO 

0930 

Land 

1000 

Setup 

1030 

1100 

1130 

1200 

Launch 

AVO_2 

AVO_2 

AVO_2  & 
MC 2 

MC_2 

1230 

AOO 

Return 

1300 

Land 

1330 

1400 

AVO 3 

1430 

MC  On 

1500 

1530 

Planning 

1600 

Launch 

1630 

Return 

AOO 

1700 

Land 

1730 

1800 

1830 

1900 

Return 

1930 

Land 

2000 

TD 

TD 

TD 

TD 

2030 

Move 

Move 

Move 

Move 

2100 
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MISSION  PROFILES  FOR  A  12-HOUR  MISSION 


I  Any  12  Hours  with  One  Hour  Gaps  Between  Missions  J 
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